Combining Audio and Video by Dominance in Bimodal Emotion Recognition1

Emotion recognition has been one of the most important issues in human computer interaction (HCI). In this paper, we propose a novel bimodal emotion recognition approach by using the boosting-based framework, in which we can automatically determine the adaptive weights for audio and visual features. In this way, we balance the dominances of audio and visual features dynamically in feature-level to obtain better performance. To ensure the tracking accuracy of facial feature points, the traditional KLT algorithm is integrated with Point Distribution Model (PDM) to guide the deformation of facial features. Experiments show the validity and effectiveness of our method.